A Study on Memory-Based Communications and Synchronization in Distributed-Memory Systems

In terms of facilities for communications and synchronization in parallel programs, the descriptive power of the shared-memory model is equal to that of the message-passing-style (send-and-receive-type) model. From the viewpoint of performance, however,the situation is different. In modern architectures for distributedmemory multicomputers, memory is an important and fundamental building-block of the nodes. It is thus directly accessed by processors that use their memory management units for protection and virtualization. Therefore, a very convenient way of improving performance or increasing the variety of functions is to have the communication/synchronization subsystems handle information related to the memory locations at which target data are stored. In more concrete terms, if communication/synchronization subsystems do so, they can use the information to reduce the number of copies of data. Moreover, these subsystems have been made capable of directly accessing data in user tasks, it becomes possible to implement advanced synchronization functions (e.g., atomic operations, queue operations and so forth), as well as simple read/write operations, within the subsystems. In short, communication/synchronization subsystems based on the shared-memory model are superior in this way to message-passing-style subsystems which simply relay data from one task to another. This conclusion holds whether the subsystem is implemented in hardware or software. In this thesis, the Memory-Based Communications and Synchronization (MBCS) scheme is proposed. In this scheme, recognition and exploitation of information on the locations of target data by communication/synchronization subsystems is proposed, along with brand-new mechanisms based on this scheme. Effectiveness of the overall scheme is then shown by experimental verification and by discussion and analysis. Going into more detail, subsystems based on the MBCS scheme are classified into three categories that correspond to the different grain sizes of data within operations and on implementation methodology. A communication/synchronization mechanism for each category is then proposed. The first mechanism, called the Memory-Based Processor (MBP), is a hardware-implemented fine-grained communication/synchronization mechanism. The MBP is also a building-block for hardware-based distributed shared memory. The second is the Memory-Based Communication Facility (MBCF) which is a software-implemented medium-grained communication/synchronization mechanism made with off-the-shelf network hardware. The third is the the Memory-Based Processor II (MBP2), a hardware-implemented medium-grained mechanism which was designed and developed on the basis of research results on the MBCF. In this thesis, (1) brand-new functions to run on these mechanisms are proposed, (2) explanations of their behaviors and of high-speed implementation methods are given, (3) comparisons are made with other existing mechanisms, qualitative discussions are presented, and (4) experimental verification is described. Through these discussions, the effectiveness of the MBCS scheme will be made clear.